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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

" !I ^"SSl ? r repl . V specified above is less tnan thirtv < 30 > davs « a re P'y w<*in U» statutory minimum of thirty (30) days will be considered timely. 

- \1 NO penod for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even rf timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)S Responsive to communication(s) filed on 15 March 2001 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) K Claim(s) 1^36 is/are pending in the application. 

4a) Of the above cJaim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) ^ Claim(s) U36 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) |3 The specification is objected to by the Examiner. 

10) KI The drawing(s) filed on 15 March 2001 is/are: a)Q accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)H None of: 

1-K^ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) [S Notice of References Cited (PTO-892) 4) interview Summary (PTO-413) 

2) Q Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) D Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) Q Notice of Informal Patent Application (PTO-152) 

Paper No(s)/Mail Date . 6) □ other . 



U.S. Patent and Trademark Office 

PTOL-326 (Rev. 1-04) 
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DETAILED ACTION 
Drawings 

1. The drawings are objected to because Figures 2-4 appear too dark to accurately 
comprehend applicant's full disclosure. Corrected drawing sheets in compliance with 37 CFR 
1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any 
amended replacement drawing sheet should include all of the figures appearing on the immediate 
prior version of the sheet, even if only one figure is being amended. The figure or figure number 
of an amended drawing should not be labeled as "amended." If a drawing figure is to be 
canceled, the appropriate figure must be removed from the replacement sheet, and where 
necessary, the remaining figures must be renumbered and appropriate changes made to the brief 
description of the several views of the drawings for consistency. Additional replacement sheets 
may be necessary to show the renumbering of the remaining figures. The replacement sheet(s) 
should be labeled "Replacement Sheet" in the page header (as per 37 CFR 1.84(c)) so as not to 
obstruct any portion of the drawing figures. If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 

Specification 

2. The disclosure is objected to because of the following informalities: 

a. Items #8 and #10 are both referred to as "processor," but drawings make a 
distinction. 

b. Item #12 is referred to using different names in specification and drawing Fig. 1 . 
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c. Item #6 is referred to in specification as "storage disk," but drawings refer to it as 
mass storage. 

d. Item #14 is not described within specification. 
Appropriate correction is required. 

e. The following guidelines illustrate the preferred layout for the specification of a 
utility application. These guidelines are suggested for the applicant's use. 

Arrangement of the Specification 

As provided in 37 CFR 1.77(b), the specification of a utility application should include 
the following sections in order. Each of the lettered items should appear in upper case, without 
underlining or bold type, as a section heading. If no text follows the section heading, the phrase 
"Not Applicable" should follow the section heading: 

(a) TITLE OF THE INVENTION. 

(b) CROSS-REFERENCE TO RELATED APPLICATIONS. 

^c) STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR 
DEVELOPMENT. 

(d) INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A 

COMPACT DISC (See 37 CFR 1.52(e)(5) and MPEP 608.05. Computer program 
listings (37 CFR 1.96(c)), "Sequence Listings" (37 CFR 1.821(c)), and tables 
having more than 50 pages of text are permitted to be submitted on compact 
discs.) or 

REFERENCE TO A "MICROFICHE APPENDIX" (See MPEP § 608.05(a). 
"Microfiche Appendices" were accepted by the Office until March 1, 2001.) 

(e) BACKGROUND OF THE INVENTION. 

(1) Field of the Invention. 

(2) Description of Related Art including information disclosed under 37 CFR 1.97 
and 1.98. 

(f) BRIEF SUMMARY OF THE INVENTION. 

(g) BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S). 

(h) DETAILED DESCRIPTION OF THE INVENTION. 

(i) CLAIM OR CLAIMS (commencing on a separate sheet). 

(j) ABSTRACT OF THE DISCLOSURE (commencing on a separate sheet). 

(k) SEQUENCE LISTING (See MPEP § 2424 and 37 CFR 1.821-1.825. A "Sequence 
Listing" is required on paper if the application discloses a nucleotide or amino 
acid sequence as defined in 37 CFR 1.821(a) and if the required "Sequence 
Listing" is not submitted as an electronic document on compact disc). 
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Claim Rejections - 35 USC § 112 



The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 



3. Regarding claims 19 and 35, the phrase "such as" renders the claims indefinite because it 
is unclear whether the limitations following the phrase are part of the claimed invention. See 
MPEP§ 2173.05(d). 



Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

4. Claims 1-36 are rejected under 35 U.S.C. 103(a) as being unpatentable over Bang et al 
(US Pat. No. 5,715,325) in view of Neff et al (US Pat. No. 5,809,171), and further in view of 
Cannon. (US Pat. No. 6,286,005). 



Application/Control Number: 09/808,4 1 5 Page 5 

Art Unit: 2612 

Claim 1 recites a method of measuring brand exposure in a video stream, comprising the 
steps of: 

a. providing a reference mask representing a trade mark; 

b. capturing frames from the video stream; 

c. searching each captured frame using the reference mask to determine a respective 
correlation value indicative of the likelihood of the presence of the trade mark of 
the mask in that captured frame in dependence upon correlation between the mask 
and part of that captured frame; and 

d. calculating a brand exposure value for the video stream in dependence upon the 
determined correlation values. 

As to sub-elements "b" and "c", Bang discloses an image detection system which 
captures a video image and determines the identity of the face within the captured frame. This is 
achieved through a comparison of the face with various templates (i.e., reference mask) stored on 
the system. The template which scores the strongest correlation (i.e., correlation value) to the 
face is determined to be the match. (Col. 1, Ln. 15-22 & Col. 2, Ln. 10-46). But, Bang fails to 
disclose the limitations contained in sub-elements "a" and "d " However, these limitations are 
contained in the references discussed below. 

As to sub-element "a", Neff discloses a similar system in which various "images" can be 
identified via the use of an image processing method and apparatus. This is achieved by 
comparing the test image with an image template stored on the system. (Abstract & Col. 1, Ln. 
12-17). The Examiner broadly interprets "image" to encompass any image, symbol, etc. stored as 
a template, including trade marks. 
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As to sub-element "d," Cannon discloses a method of analyzing advertising data, which 
allows an advertiser to evaluate, create, rank, and score advertising campaigns based upon ad 
exposure (i.e., calculate an exposure value). (Col. 3, Ln. 31-67 ; Col. 29, Ln. 35-67 thru Col. 30, 
Ln. 1-19). (Essentially, this is the function of the applicant's invention—to provide advertisement 
related statistics via the analysis of ad exposure). As disclosed by Cannon, various methods of 
ad evaluation via ad exposure are well-known in the art. Applicant is only claiming an 
alternative method of ascertaining ad exposure achieved through image recognition, which is 
also well known in the art. (as evidenced by Bang). 

Accordingly, it would have been obvious to one of ordinary skill in this art at the time of 
applicant's invention to combine the systems of Bang, Neff, and Cannon in order to provide a 
system capable of determining the advertising exposure/value of advertising campaigns via the 
calculation of trade mark exposure, thereby providing business entities with an alternative 
method of evaluating advertisement expenditures (Cannon, col. 3 In. 60-67.) 

Claims 18, 19, and 36 correspond to the method claim 1. Thus, each is analyzed and 
rejected as previously discussed. 

Claim 2 recites the method of claim 1, wherein the determined correlation value for each 
frame has either of only two possible values. As discussed above, the combined systems of 
Bang, Neff, and Cannon contain all limitations of claim 1, and Bang further teaches the system 
can calculate multiple numbers of possible correlation values. (Col. 7, Ln. 4-39). In view of this 
disclosure, the limitation of claim 2 is an obvious variant. In other words, a system, which 
calculates only two possible values, is an obvious variant of one that calculates two or more 
possible values. Accordingly, the combined systems of Bang, Neff, and Cannon contain the 
limitations of claim 2. 
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Claim 20 corresponds to the method claim 2. Therefore, it is analyzed and rejected as 
previously discussed. 

Claim 3 recites the method of claim 1, wherein the determined correlation value for each 
frame has one of a multiplicity of possible values. As discussed above, the combined systems of 
Bang, Neff, and Cannon contain all limitations of claim 1, and Bang further teaches the system 
can calculate multiple numbers of possible correlation values. (Col. 7, Ln. 4-39). Accordingly, 
the combined systems of Bang, Neff, and Cannon contain the limitations of claim 3. 

Claim 21 corresponds to the method claim 3. Therefore, it is analyzed and rejected as 
previously discussed. 

Claim 4 recites the method of claim 3, wherein, in the calculating step, such a determined 
correlation value is taken into account only if that determined correlation value lies within a 
particular range of the values. As discussed above, the combined systems of Bang, Neff, and 
Cannon contain all limitations of claim 3, and Bang further teaches that only templates that 
exceed predetermined scores are further analyzed. (Col. 7, Ln. 4-39). Accordingly, the 
combined systems of Bang, Neff, and Cannon contain the limitations of claim 4. 

Claim 22-corresponds to the method claim 4. Therefore, it is analyzed and rejected as 
previously discussed. 

Claim 5 recites the method of claim 4, wherein, in the calculating step, the magnitude of 
such a determined correlation value within said range is taken into account. As discussed above, 
the combined systems of Bang, Neff, and Cannon contain all limitations of claim 4, and Bang 
further teaches that only templates (within the specified range), with the highest correlation value 
are selected (i.e., taken into account their magnitudes). (Col. 7, Ln. 4-39). Accordingly, the 
combined systems of Bang, Neff, and Cannon contain the limitations of claim 5. 
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Claim 23 corresponds to the method claim 5. Therefore, it is analyzed and rejected as 
previously discussed. 

Claim 6 recites the method of claim 1, wherein each searching step includes the step of 
determining a respective scale value in dependence upon the scale of said part of the captured 
. frame relative to the mask; and in the calculating step, the brand exposure value is calculated in 
dependence upon both the determined correlation values and the respective scale values. As 
discussed above, the combined systems of Bang, NefF, and Cannon contain all limitations of 
claim 1, and Neff further teaches that the scale (i.e., size) of the captured image is taken into 
account when determining the overall correlation value. (Col. 14, Ln. 8-44). Accordingly, the 
combined systems of Bang, NefF, and Cannon contain the limitations of claim 6. 

Claim 24 corresponds to method claim 6. Thus, it is analyzed and rejected as previously 
discussed. 

Claim 7 recites the method of claim 1, wherein: each searching step includes the step of 
determining a respective position value in dependence upon the position of said part of the 
captured frame relative to the complete frame; and in the calculating step, the brand exposure 

value is calculated in dependence upon both the determined correlation values and the respective 

determined position values. As discussed above, the combined systems of Bang, NefF, and 
Cannon contain all limitations of claim 1, and NefF further teaches the position oF the test image 
is also taken into account when determining the overall correlation value. (Col. 14, Ln. 31-44). 
Accordingly, the combined systems oFBang, NefF, and Cannon contain the limitations oF claim 
7. 

Claim 25 corresponds to the method claim 7. ThereFore, it is analyzed and rejected as 
previously discussed. 
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Claim 8 recites the method of claim 1, wherein the method further includes the step of 
providing an audience rating value; and in the calculating step, the brand exposure value is 
calculated in dependence upon both the determined correlation values and the audience rating 
value. As discussed above, the combined systems of Bang, Neff, and Cannon contain all 
limitations of claim 1, and Cannon further teaches it is well-known in this art to use consumer 
viewing statistics (i.e., audience rating values) to assist in the evaluation of advertising 
campaigns. (Col. 1, Ln. 64-67 thru Col. 2, Ln. 1-16). Accordingly, it would have been obvious 
to one of ordinary skill in this art at the time of applicant's invention to further modify the 
combined systems of Bang, Neff, and Cannon to allow the system to also consider audience 
ratings in order to provide a more effective method of evaluating advertisement decisions. 

Claim 26 corresponds to the method claim 8. Therefore, it is analyzed and rejected as 
previously discussed. 

Claim 9 recites the method of claim 8, wherein the provided audience rating value varies 
for different frames; and in the calculating step, the brand exposure value is calculated in 
dependence upon both the determined correlation values and the respective audience rating 
■"values: — As discussed above," the combined systems of Bang, -Neff ,~and~Gannon contain alL 
limitations of claim 8, and Bang further teaches his system (and any other image recognition 
apparatus) is capable of capturing framed images in a video stream. (Col. 1, Ln. 15-22 & Col. 2, 
Ln. 10-46). Cannon further teaches that viewer statistics can be collected on a minute-by-minute 
basis, since of course, viewer preferences can change over time. (Col. 6, Ln. 15-21). During 
these minute-by-minute intervals, frames are being transmitted. Therefore, it logically follows 
that certain frames will have higher viewer preferences than others. Thus, the combined 
teachings of Bang and Cannon would suggest a system capable of analyzing viewer preferences 
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based upon captured frames (since audience rating value varies for different frames). 
Accordingly, it would have been obvious to one of ordinary skill in this art at the time of 
applicant's invention to further modify the combined teaching of Cannon and Bang in order to 
provide a system capable of analyzing captured frames in order to ascertain audience rating 
values, thereby providing a real-time calculation of advertisement effectiveness. 

Claim 27 corresponds the method claim 9. Therefore, it is analyzed and rejected as 
previously discussed. 

Claim 10 recites the method of claim 1, wherein, in each searching step, in the case of a 
plurality of presences of the trade mark of the mask in the frame, the method is capable of 
determining a plurality of correlation values for that frame. This claim is an obvious variant of 
claim 1. Under the rejection of claim 1, the combined references teach the system is capable of 
detecting whatever target images are present in the frame. This teaching is interpreted to 
encompass one or various images in the viewing space; therefore, the system is capable of 
calculating correlation values for a plurality of images. Accordingly, the combined systems of 
Bang, Neff, and Cannon contain all limitations of claim 10. 

Claim 28 corresponds to-the method claim 10 -Therefore, it is analyzed and rejected as 

previously discussed. 

Claim 1 1 recites the method of claim 1 wherein the reference mask is one of a plurality of 
different such reference masks representing the same trade mark; and such searching steps are 
performed for each of the reference masks. As discussed above, the combined systems of Bang, 
Neff, and Cannon contain all limitations of claim 1, and Neff further teaches the templates can be 
warped or textured (i.e., plurality of reference masks representing the same trade mark) in order 
to compensate for differences between the test image and the template. Here, the templates are 
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divided and each is compared. (Col. 6, Ln. 4-28). Accordingly, the combined systems of Bang, 
Neff, and Cannon contain all limitations of claim 11. 

Claim 29 corresponds to the method claim 11. Thus, it is analyzed and rejected as 
previously discussed. 

Claim 12 recites the method of claim 11, wherein a respective mask weighting value is 
provided for each reference mask; and in the calculating step, the brand exposure value is 
calculated in dependence upon both the determined correlation values and the respective mask 
weighting values. In applicant's specification, the respective mask weighting value can 
determined as a proportion of size, definition, or contrast of the image. (Spec. Pg. 4, Ln. 22-27). 
As discussed above, the combined systems of Bang, Neff, and Cannon contain all limitations of 
claim 11, and Neff further teaches that size is used as a factor in determining the correlation 
value. (Col. 14, Ln. 31-44). Accordingly, the combined systems of Bang, Neff, and Cannon 
contain all limitations of claim 12. 

Claim 30 corresponds to the method claim 12. Thus, it is analyzed and rejected as 
previously discussed. 

-- Claim 13 recites the method of claim 1, wherein the reference mask is one of a plurality 
of different such reference masks representing different trade marks; such searching steps are 
performed for each of the reference masks; and in the calculating step, a plurality of brand 
exposure values are calculated for the different trade marks. As discussed above, the combined 
systems of Bang, Neff, and Cannon contain all limitations of claim 1, and Bang further teaches 
the system is capable of storing multiple templates, which represent different faces (i.e., objects). 
(Col. 3, Ln. 10-29). In light of this disclosure, it is inherent the system be capable of calculating 
different exposure values for the different templates which possible correspond to various test 
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images located in the frame. Accordingly, the combined systems of Bang, Neff, and Cannon 
contain all limitations of claim 13. 

Claim 31 corresponds to the method claim 13. Therefore, it is analyzed and rejected as 
previously discussed. 

Claim 14 recites the identical limitations of claim 12, except it depends from claim 13, 
not claim 11. However, the same analysis and rejection used under claim 12 is applicable here. 
Accordingly, the combined systems of Bang, Neff, and Cannon contain all limitations of claim 
14. 

Claim 32 corresponds to the method claim 14. Thus, it is analyzed and rejected as 
previously discussed. 

Claim 15 recites the method of claim 1, further comprising the step of storing each of the 
frame which contributes to the brand exposure values; and storing the respective value(s) 
determined from that frame. As discussed above, the combined systems of Bang, Neff, and 
Cannon contain all limitations of claim 1, and Neff further teaches the correlation values are 
stored on the system. (Col. 15, Ln. 59-67). Also as discussed above, storing frames is inherent 
- in light of the disclosure of the combined references. In order to analyzed the captured images; 
the system must be capable of storing those images. Accordingly, the combined systems of 
Bang, Neff, and Cannon contain all limitations of claim 15. 

Claim 33 corresponds to the method claim 15. Therefore, it is analyzed and rejected as 
previously discussed. 

Claim 16 recites the method of claim 1, wherein the calculating step comprises summing 
the correlation values or a function of each correlation value. As discussed above, the combined 
systems of Bang, Neff, and Cannon contain all limitations of claim 1, and Neff further teaches 
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the system is capable of summing correlation values. (Col. 18, Ln. 20-39). Accordingly, the 
combined systems of Bang, Neff, and Cannon contain all limitations of claim 16. 

Claim 34 corresponds to the method claim 16. Therefore, it is analyzed and rejected as 
previously discussed. 

Claim 17 recites limitations to numerous to detail herein. All but one limitation in claim 
17 is encompassed within claim 1, and therefore, claim 17 is analyzed and rejected as previously 
discussed therein. The additional limitation in Claim 17 recites "searching each captured frame 
image to identify if... the mask is above a predetermined threshold." Bang teaches the use of 
thresholds in determining a facial match. (Abstract). Accordingly, the combined systems of 
Bang, Neff, and Cannon contain all limitations of claim 17. 

Claim 35 corresponds to the method claim 17. Therefore, it is analyzed and rejected as 
previously discussed. 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure _ 

a. Takemura et al (US Pat. No. 5,239,595) disclose an optical method of recognizing 
a pattern. 

b. Trew et al (US Pat. No. 5,561,718) disclose a system capable of identifying a 
image in a video stream. 

c. Mukohzaka (US Pat. No. 5,878,157) discloses an image identification apparatus. 

d. Steffens et al (US Pat. No. 6,301,370) disclose an apparatus capable of face 
recognition in a video image. 
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6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jade O. Laye whose telephone number is (571) 272-7303. The 
examiner can normally be reached on Mon. 7:30am-4, Tues. 7:30-2, W-Fri. 7:30-4. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Miller can be reached on (571) 272-7353. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-2 1 7-9 1 97 (toll-free). 

Examiner's Initial's 

March 10 th , 2005. 
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